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Abstract 



Positive definite (p.d.) matrices arise naturally in many areas within mathemat- 
ics and also feature extensively in scientific applications. In modern high-dimensional 
applications, a common approach to finding sparse positive definite matrices is to 
threshold their small off-diagonal elements. This thresholding, sometimes referred to 
as hard-thresholding, sets small elements to zero. Thresholding has tlK> attractive prop- 
erty that the resulting matrices are sparse, and are thus easier to interpret and work 
with. In many applications, it is often required, and thus implicitly assumed, that 
thresholded matrices retain positive definiteness. In this paper wc formally investigate 
the algebraic properties of p.d. matrices which are thresholded. We demonstrate that 
for positive definiteness to be preserved, the pattern of elements to be set to zero has 
to necessarily correspond to a graph which is a union of disconnected complete com- 
ponents. This result rigorously demonstrates that, except in special cases, positive 
definiteness can be easily lost. Wo then proceed to demonstrate that the class of di- 
agonally dominant matrices is not maximal in terms of retaining positive definiteness 
when thresholded. Consequently, we derive characterizations of matrices which retain 
positive definiteness when thresholded with respect to important classes of graphs. In 
particular, we demonstrate that retaining positive definiteness upon thresholding is 
governed by complex algebraic conditions. 
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1 Introduction 

Positive definite matrices arise naturally in various settings. Concrete examples are found 
in the fields of probability, statistics and machine learning. Here they are often used to rep- 
resent covariance (or correlation) matrices. These covariance matrices encode multivariate 
relationships in a random vector and feature prominently in procedures like principal com- 
ponent analysis (PC A), linear discriminant analysis (LDA), etc.. Unlike Euclidean space, 
the open cone of positive definite matrices is more difficult to analyze because of the complex 
relationships between the elements of the matrices. In modern high-dimensional applica- 
tions, a common approach to finding sparse positive definite or sparse covariance matrices 
is to hard-threshold small off-diagonal elements of given positive definite matrices. For ex- 
ample, a common approach to finding statistically significant relationships between genes 
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in a gene-gene interaction analysis is to threshold the corresponding correlation matrix of 
gene-gene associations (see [B], [S]). Thresholding has the property of setting correlations 
which are small in absolute value to zero. This procedure therefore eliminates spurious 
correlations which could have risen purely due to random noise. In the process, only statis- 
tically significant correlations are retained. This thresholding approach has the attractive 
property that it is easily computed, i.e., it is a highly scalable procedure, and yields a smaller 
number of large correlation coefficients. The resulting thresholded matrices are sparse and 
are thus easier to interpret and manipulate. Furthermore, finding a regularized or sparse 
estimate of the correlation matrix is often undertaken with an ulterior goal in mind. In- 
deed, these thresholded matrices are used as regularized estimates of covariance matrices in 
various applications. In particular, they are ingredients in several procedures in statistics 
and machine learning, such as principal component analysis, linear discriminant analysis, 
canonical correlation analysis, etc. (see [J, [7]). For these procedures to be widely applica- 
ble, it is often assumed implicitly that the thresholded matrices retain positive definiteness. 
The theoretical investigation of whether this is true is critical for the validity of such appli- 
cations. Statistical properties of thresholded correlation matrices have been studied in the 
literature (see [1] ) , however, algebraic properties of such matrices have not been previously 
investigated. 

In this paper, we provide a rigorous answer to the aforementioned questions. In partic- 
ular, we prove that, if positive definiteness is to be retained after thresholding, the pattern 
of elements to be thresholded has to necessarily correspond to a graph which is a union of 
disconnected complete components. We also consider the problem of thresholding positive 
definite matrices at a given level e, i.e., only to zero out elements which are less than e in 
magnitude. In this case, we prove that only matrices which already have zeros according 
to a tree, will retain positive definiteness for all values of the thresholding parameter e. 
These results imply that positive definiteness can be easily lost due to hard-thresholding. 
Furthermore, it is well known that the class of diagonally dominant matrices with strictly 
positive diagonal entries is positive definite. Procedures such as hard-thresholding maintain 
diagonal dominance. Hence, we also investigate if positive definiteness is only retained by 
the class of diagonally dominant matrices, and proceed to demonstrate that this class is not 
maximal. We therefore proceed to identify algebraic relationships which characterize retain- 
ing positive definiteness for important classes of graphs. Our results formally demonstrate 
that thresholding approaches used in the literature can lead to sparse matrices which are 
no longer positive definite. 

As a concrete example, let us threshold the (2, 3) and (3, 2) elements of the following 
symmetric matrix 



One can easily check that A is positive definite, but A' is not. So a fundamental question 
that arises is, when do we know for certain that A' > 0. If A comes from certain special 
subclasses of the cone of p.d. matrices, the answer is immediate. For instance, ii A E B 
where B C P"*" is the class of diagonally dominant matrices, then A' is always positive 
definite. As pointed out in the example above, this property is not true in general for an 
arbitrary positive definite matrix A. 

More formally, the elements to threshold are naturally encoded in a graph G — {V, E) 
with V = {1, . . . ,p} and with the convention that Aij is set to zero if and only if (i, j) ^ E. 
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We denote the thresholded matrix by Aq- So when will Aq > for certain, i.e., what classes 
of graphs will ensure positive definiteness. It is not immediately clear what G should be. 
In this paper we give results of potentially great consequence, that only for a narrow class 
of graphs can we ensure positive definiteness. The statement of the main theorem in this 
paper is given below. 

Theorem 1.1. Let A be an arbitrary symmetric matrix such that A > 0, i.e., A g P"*". 
Threshold A according to a graph G = (V, E) with the resulting thresholded matrix denoted 
by Aq . Then 

T 

> for any AeV+ <^ G =[jG, for some r e N, 

1=1 

where Gi, i = 1, . . . ,t, denote disconnected and complete components of G. 

The (<J=) part of the theorem is intuitive and straightforward but the (^) part does 
come as somewhat of a surprise. It is also a stark reminder that indiscriminate or arbitrary 
thresholding of a positive definite matrix can quickly lead to loss of positive definiteness. 

The remainder of the paper is structured as follows. Section [2] introduces basic terminol- 
ogy, notation and preliminaries. Section [3] provides a proof of the main theorem and further 
extensions. In particular, we consider the effect of thresholding when a positive definite 
matrix already has zeros according to a graph G. We also investigate the impact of thresh- 
olding when only elements smaller in magnitude than a prescribed value are thresholded, 
as is often done in practice. We also study the maximal class of matrices retaining posi- 
tive definiteness and show that the set of diagonally dominant matrices is never maximal. 
In Section [4] we characterize the class of matrices that retain positive definiteness when 
thresholded with respect to important classes of graphs. We discover that, even in simple 
cases, characterizing the maximal class leads to complicated algebraic relations. There is 
thus little hope of obtaining a characterization in terms of a simple property like diagonal 
dominance. 

2 Preliminaries 
2.1 Graph theory 

Let G = {V,E) be an undirected graph with vertex set V = {1, . . . ,p} and edge set E. 
Two vertices a,h € V ^ a ^ b, are said to be adjacent in G if (a, 6) G E. We say that 
the graph G" = [V',E') is a subgraph of G = {V,E), denoted by G" C G, if C y and 
E' C E. In addition, if G' C G and E' ^ V x V E, we say that G' is an induced 
subgraph of G. We shall consider only induced subgraphs in what follows. For a subset 
A C V, the induced subgraph Ga — (A, A x An E) is said to be the graph induced by 
A. A graph G is called complete if every pair of vertices are adjacent. A clique of G is an 
induced complete subgraph of G that is not a proper subset of any other induced complete 
subgraphs of G. A path of length fc > 1 from vertex i to j is a finite sequence of distinct 
vertices uq = «,..., Wfc = j in and edges {vq, vi), . . . , Ufc) € E. An n-cycle in G is a 

path of length n with an additional edge connecting the two end points. A graph G is called 
connected if for any pair of distinct vertices i,j € V there exists a path between them. 

Now let A, B,C C V he three nonempty subsets of V. We say that G separates A from 
B if every path from a vertex a S A to a vertex b € B contains a vertex in C. The graph 
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G is said to be chordal or decomposable (|2], [3]) if it does not contain a cycle of length > 4 
as an induced subgraph. Alternatively, a graph G is said to be chordal or decomposable if 
either G is complete or if there exist subsets A, B,C C V such that 

1. V ^ AU BUG; 

2. G separates A from B; 

3. C is complete; 

4. Gyiuc and Gsuc are decomposable. 

A triple {A, B, G) satisfying the first three properties above is said to be a decomposition of 
G. 

Furthermore, let Gi, G2, . . . , Gfe be an ordering of the cliques of a graph G. Define the 
history up to clique q as Hq = Gi U • • • U G^ and the separator Sq = GqCiHq-i. The ordering 
{Gi, . . . , Gfc} is said to be a perfect ordering if for every g = 2, . . . , fc, there exists p < q — 1 
such that Sq C Gp. A well known result in graph theory is that every decomposable graph 
admits a perfect ordering of its cliques. Conversely, this property characterizes decomposable 
graphs. A special class of decomposable graphs are trees. These are connected graphs on n 
vertices with exactly n — 1 edges. A tree can also be defined as a connected graph with no 
cycle of length n > 3, or a connected graph with a unique path between any two vertices. 
Alternatively, a tree is a connected decomposable graph with maximal clique size 2. 

In the remainder of the paper, we will assume that the vertices ^ of G are labeled by 
the set {1, 2, . . . , n}. As mentioned earlier, the graph induces a hard-thresholding operation, 
mapping every symmetric matrix A = (aij) G M„ to a matrix Ac in Mn defined by 

, , _ / a,j if {i,j) £E or i=j 
(^G)ij <y Q otherwise 

We say that the matrix Aq is obtained from A by thresholding A with respect to the graph 
G. 

2.2 Linear Algebra 

We denote by P+ the cone of positive matrices. Let G = {V. E) be a graph and let 

Pg = {A e P+ : a,j = if ^E,i^ j) 

be the set of positive definite matrices with fixed zeros according to G. 

A class of matrices which are guaranteed to retain positive definiteness upon thresholding 
by any arbitrary graph is the class of diagonally dominant matrices. We formalize this 
concept in the following well known results from linear algebra [4j. 

Definition 2.1 (Strictly Diagonally Dominant matrices). A matrix A is said to be strictly 
diagonally dominant if for every z = 1, . . . , n, 

Tiieorem 2.2 (Gershgorin circle theorem, [i] - Theorem 6.1.1). Let A = (a,y) e Afp(M). 
For i — 1, . . . ,p, let Ri = '^j^i o-f^d let Di = D^ua, Ri) be the closed disc with center 
an and radius Ri . Then every eigenvalue of A belongs to at least one of the discs Di . 
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Corollary 2.3 (Positive definiteness of diagonally dominant matrices). A strictly diagonally 
dominant matrix with positive diagonal entries is necessarily positive definite. 



Corollary 2.4 (Positive definiteness of thresholdcd strictly diagonally dominant matrices). 
Let G be an arbitrary graph and consider a strictly diagonally dominant matrix A. Then Aq 
is positive definite. 

The proof follows immediately from the Gershgorin circle theorem. 

A natural question is whether positive definiteness is in general retained when matrices 
are thresholded. Moreover, are diagonally dominant matrices the maximal class which 
retains positive definiteness when thresholded ? We investigate these and related questions 
in this paper. 

Finally, recall that a (pi +P2) x {p\ +P2) symmetric block matrix 



where A G Mp^xpi,B e Mp^xpa' ^ ^ Mp^xp2, is positive definite if and only if D 
is positive definite and Si = A — BD~^B* is positive definite. The matrix Si is called 
the Schur complement of D in M. Alternatively, M is positive definite if and only if A is 
positive definite and S2 = D — B*A~^B is positive definite. The matrix ^2 is called the 
Schur complement of A in M. 

3 Thresholding of arbitrary positive definite matrices 

We now proceed to prove the main result of this paper. 

Theorem 3.1. Let A be an arbitrary symmetric matrix such that A > 0, i.e., A G P+. 
Threshold A with respect to the graph G = {y,E) with the resulting thresholded matrix 
denoted by Aq. Then 



where Gi, i 1, . . . ,t, are disconnected and complete components of G (or equivalently G 

is comprised of only complete disconnected components). 

Proof. (^) This direction is straightforward. Note that every principal submatrix of a 
positive definite matrix is also positive definite. So Aq can be rearranged in a block diagonal 
form with each block Ai > corresponding to each subgraph G,. It follows easily that 



(=>) We shall prove the contrapositive form, i.e., for a graph G which is not a union of 

complete disconnected components, there exists a matrix A G P"*" such that Aq :^ 0. We 
shall use mathematical induction on \V\ = k. Suppose first that fc = 3. We only need to 
check that the following graph, often termed as the As graph, can lead to a loss of positive 
definiteness: 




r 




for some r G N, 



Ag > 0. 



5 



Consider the following example: 



/4 3-3\ /43-3\ 
A = I 3 4 -1 Ag= [ 3 4 . 
\ -3 -1 4 / \ -3 4 / 

One can easily check that A> 0, but Aq is not. The theorem is hence true for k = 3. 

Let us now assume the inductive hypothesis when k = p—1, i.e., let G = (V, E) be such 
that \V\=p-l. Then 

r 

Ag>0 for any A(p_i)x(p_i) e P+_i ^ G = [J 

i=l 

where Gi are complete disconnected components. Let us now show that the result holds true 
when k = p. Let G' = {V, E') be an arbitrary graph such that \V'\ = p. Let G = (V, E) be 
the subgraph of G' induced by the vertices 1}. Ifapxp positive definite matrix 

A' retains positive dcfinitcncss when thrcsholdcd with respect to G", then the (p — 1) x (p — 1) 
principal submatrix of A' will also be positive definite when thresholded with respect to G. 
Therefore, if every matrix A' G P+ stays positive definite when thresholded with respect 
to G' then, by the inductive hypothesis, the graph G can be decomposed as a union of 
complete disconnected components. So now on, let us assume that G is a union of complete 
disconnected components and assume to the contrary that G' is not a union of complete 
disconnected components. Since G' is not a union of complete disconnected components, 
vertex p can either a) be connected to exactly one complete connected component Gi of G, 
where Gi U {p} is not a complete subgraph of G' or b) to two or more connected components 
of G. In the first case, there exists vertices x,y G Ci such that {p, x, y} induces the following 
subgraph from G': 



X 




In the second case, there exists vertices x € Ci and y € C2, where Gi and G2 are different 
connected components of G, such that {p, x, y} induces the following subgraph from G' 



Let us construct A' > as follows 



p X y 



( a 



^xp ^xx CLxy 
CLyp CLyx ^yy 



V : : ; 

where the 3 by 3 upper left block corresponds to the example above for the fc = 3 case. It 
is possible to choose the other entries of A' such that A' > {a. concrete example can be 
constructed by augmenting the 3 by 3 submatrix with a block diagonal identity matrix of 
dimension p — 3). By the fc = 3 case, when A' is thresholded with respect to G", the 3 by 
3 block loses positive definiteness, and consequently A'q, 0. This yields a contradiction 
to the initial hypothesis that A'q, > 0. Therefore, the graph G' decomposes as a union of 
complete disconnected components. □ 

Remarks: 

1. An alternative proof, which circumvents the inductive argument, would follow from 
first proving that the A^ graph is necessarily an induced subgraph of G' when G" is 
not a union of complete disconnected components. The remainder of the proof would 
follow the same line of argument as above. 

2. Note that the proof for any arbitrary fixed dimension hinges on losing positive defi- 
niteness in the p = 3 case. 

3. The result above holds true if we replace the class of positive definite matrices by the 
class of positive semidefinite matrices since the former set is a subset of the latter. 
Also, the result remains valid if we restrict the set of positive semidefinite matrices to 
the subclass which are singular (i.e. matrices which are positive semidefinite but are 
not positive definite). 

We formalize the extension of our results to the class of singular positive semidefinite 
matrices in the following corollary. 

Corollary 3.2. Let A be an arbitrary symmetric matrix such that A > and detA — 0. 
Threshold A according to a graph G = (V, E) with the resulting thresholded matrix denoted 
by Ac ■ Then 

T 

Ag ^ for any A > with det ^ ^ ^ G = y Gi for some T e N, 

1=1 

where Gi, i = 1, . . . ,t , are disconnected and complete components of G (or equivalently G 
is comprised of only complete disconnected components). 

Proof. {■^) This direction is clear since a principal submatrix of a positive semidefinite ma- 
trix is positive semidefinite. 

We shall prove the contrapositive form. Suppose G is not a union of complete discon- 



nected components. According to Theorem 3.1 there is a matrix A > such that Aq is not 



positive definite. Let Ai and rji be the smallest eigenvalues of A and Aq respectively. Note 
that Ai > but rji < 0. Consider the matrix B ^ A — Xil. This matrix satisfies B > and 
det B = 0. In addition, Bq = Aq — Ai/ and so the smallest eigenvalue of Bq is 771 — Ai < 0. 
Hence there exist a matrix B > with det B — Q such that Bq ^0. □ 
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Theorem |3. 1| demonstrates that ensuring positive definiteness after thresholding for the 
whole cone P"*" is rather difScult; in the sense that only when G is a union of complete 
disconnected components can we ensure positive definiteness of the resulting thresholded 
matrix. It is therefore natural to try to threshold a smaller class of matrices than the whole 
space of positive definite matrices to see if positive definiteness is retained. Consider for 
example the case where a positive definite matrix A already has zeros according to a graph 
G. This will be the case, for example, when A is a sparse positive definite matrix. If A 
where to be thresholded, will it remain positive definite ? Specifically, for which subgraphs 
H oi G will Ah retain positive definiteness ? The next theorem significantly generalizes 



Theorem 3.1 and demonstrates that the answer is essentially the same. 



Theorem 3.3. Let G — (V, E) be an undirected graph and let H = {V, E') 
of G i.e., E' C E. Then Ah > for every AeVc if and only if H = GiU 
Gi, . . . , Gk are disconnected induced subgraphs of G. 



be a subgraph 
■ ■ U Gk where 



Proof. The (-4=) part of the theorem is clear since a principal submatrix of a positive definite 
matrix is positive. 

(=>) We shall prove the contrapositive form. Suppose H cannot be written as a union of 
disconnected induced subgraphs of G. Let Hi, . . . , iJ,„ be the connected components of H. 
Since H = Hi U • • • U iJ,„ and the components are disconnected, one of the Hi is not an 
induced subgraph of G. Without loss of generality, assume Hi is not an induced subgraph. 
Then there are two vertices u,v G Hi such that {u,v) ^ E' , but {u,v) G E. Since Hi is 
connected, there is a path u = vi, . . . ,Vk — v connecting u and v in Hi. Let G be the 
cycle obtained by adding the edge (u, v) to the end of this path. This cycle belongs to G. 
Therefore, when the matrix A is thresholded with respect to H, a cycle belonging to G is 
broken. In the remainder of the proof, we will prove that, for any cycle G„ of length n > 3, 
there is a positive definite matrix A — Ac^ such that A loses positive definiteness when 
thresholded by removing an edge of G„. This together with the fact that any principal 
submatrix of a positive definite matrix is also necessarily positive definite will prove the 
theorem. 

Let G„ be a cycle of length n > 3. Consider the matrix 



Aj, 



a 


1 








... 





a 


1 


2 


1 





... 











1 


2 


1 


... 
















■■. 2 


1 

















1 


2 


b 


a 











... 


b 





where the upper left block is the matrix of a path of length n - 
equal to 1 and diagonal elements equal to 2 except the (1,1) 
Clearly, An = {An)c„- We first claim that 



- 1 with off-diagonal elements 
element which is equal to a. 



detA„ = -{n-2)f3 + {n-l)a/3+{-iy'+'2ab~{ 
We will prove this formula by induction on n. If n 

A. = 



-l)a2 + (n-3)6^-(n-2)a&2. (3.1) 
3, the matrix reduces to 
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and 

det A3 = a(2/3 - b^) -{13- ab) + a{b - 2a) 



-13 + 2al3 + 2ab - 2a^ - ab^. 



So formula (3.1 1 is valid for n = 3. Suppose, by the inductive hypothesis, that formula (3.1 ) 
is valid for k = n — 1, and now consider the matrix An- The determinant of An is given by 
/3 times the determinant of the Schur complement S of the lower right block. Notice that 
the matrix S is equal to An-i with a replaced by a — a^//3, /3 replaced by 2 — 6^//3, b by 1, 
and a by —ab/j3. Therefore, by the induction hypothesis, 

detS ^ -(„-3,(2-^)+(„-2)(a-2;)(2-^)+(-ir.2| 

^.a^b"^ , ,s , f a^ 

- (n-2)^ + (n-4)-(n-3)(^a-- 

= 2 - n + a(n - 1) + ^(71 - 3 - a{n - 2)) - "-(n - 1) + 2(-l)"+i^. 

P P P 

Hence, 

det ^„ = /3 det 5 = -(n - 2)/3 + (n - \)a^ + (-l)"+i2a6 - (n -\)a^ + (n- 3)6^ -(n- 2)ab^, 



and so, formula (3.1 ) is valid for k — n. The expression in (3.1 1 is therefore valid for every 
n > 3. 

Now, let M„ = M„(a,6) be the matrix A„ with a — /3 — 2. We will prove that we can 
choose a and b such that M„ is positive definite, but Af„ is not positive definite if a is set 
to 0. 

By using the formula for det^„, it is easily shown that the determinant of the k-th 
principal submatrix, corresponding to the upper left corner of M„, is equal to 

-2{k - 2) + 4(fc - 1) + (/c - 3) - 2(/c - 2) = 1 + fc > for fc = 1, . . . , n - 1. 

Therefore, the upper left (ri — 1) x (n — 1) block of Af„ is always positive definite. The 
determinant of M„ is given by 

p{a, b)^2n-{n- l)a'^ - {n - 1)6^ + (_l)"+i2a& 

Note that p{a,b) is a concave quadratic equation in a for fixed b. If the (l,n) and (n, 1) 
elements are thresholded, i.e., if a is set to 0, the new determinant is given by 

p(0,6) := q{b) = 2n-{n-l)b^. 

So in order for the thresholded matrix to lose positive definiteness, we must choose b such 
that q{b) < i.e., 6^ > 2n/{n - 1). Hence let 

b'^ 2n 



1 

where e > 0. We claim that, if e is small enough, we can choose a such that p{a, b) > 0. 
Indeed, the discriminant of p (seen as a function of a), when the above value of b^ is 
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substituted, is given by 



A = 46^+ 4(n - 1) [2n - (n - l)b^] 
= 4(^-^+e^ -4(n-l)2e. 

Hence we can choose e > small enough such that A > 0. With this choice of e, the 
polynomial p(a, b) has two real roots and so we can choose a such that p{a, b) > 0. Therefore, 
for these choices of a and 6, the matrix M„ is positive definite, but loses positive definiteness 
if a is thresholded to zero. The above arguments provide the needed example for the cycle 
C„ and therefore proves the theorem. □ 



Theorem |3.3| in fact also yields the result of Theorem |3.1| as a special case. Corollary |3.4| 
below gives a formal proof of this claim and thus provides a better understanding of Theorem 
|3.1[ In addition. Corollary [33] below aims to characterize the class of sparse positive definite 
matrices which retain positive definiteness regardless of the thresholding subgraph. 

Corollary 3.4. Let G be an undirected graph. Assume Aq > for every ^ > 0. Then G 
is a union of complete disconnected components. 



Proof. Apply Theorem 3.3 with G equal to a complete graph and H equal to the graph G 



specified in the statement of the corollary. □ 

Corollary 3.5. Let G he an undirected graph. Then Ah > for every matrix A g Pq and 
every subgraph H of G if and only if G is a union of trees. 

Proof. (■<=) This follows from the fact that every subgraph of a tree is a union of disconnected 
subtrees. 



(=>) From Theorem 3.3 every subgraph of G must be a union of disconnected induced 
subgraphs. Therefore G does not contain any cycle of length > 3 and thus G is a union of 
trees. □ 



Corollary |3.5| states that if we start with a positive definite matrix A that already has 
zeros according to a tree, then any further thresholding of A will retain positive definiteness. 

The results above consider thresholding of elements regardless of their magnitude. In 
practical applications however, hard-thresholding is often performed on the smaller elements 
of the positive definite matrix in order to induce sparsity. A more natural question therefore 
would be to ask: if we threshold rather the small elements of a positive definite matrix, would 
the new thresholded matrix be positive definite ? We show below that this is possible in 
general only if the original matrix has zeros according to a tree. 

First, we introduce some notation. We will say that the matrix B is the hard-thresholded 
version of A at level rj if bij ~ Oij when |a.y | > rj or i = j, and bij = otherwise. 

Theorem 3.6. Let G be an undirected graph with n > 3 vertices and consider ¥g, the class 
of positive definite matrices with zeros according to G. The following are equivalent: 

1. There exists rj > such that the hard-thresholded version of A at level rj is positive 
definite for every A € Pg; 

2. G is a tree. 
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Moreover, if G is a tree, the hard-thresholded version of A at level rj is positive definite for 
every rj > 0. 



Proof. (2) (1) If G is a tree, we already know from Corollary 3.5 that every A e Pq will 
retain positive definiteness when thresholded with respect to any subgraph. Therefore, the 
thresholded matrix will always be positive definite regardless of the value of r/. 
(1) ^ (2) We will prove the contrapositive form. Suppose G is not a tree. Then G contains 
a cycle of length n > 3. To prove the result, we need to show that for any level rj > 0, there 
exists a matrix A > with zeros according to G such that the thresholded version of A at 
level rj is not positive definite. It is therefore sufficient to provide a matrix A S Pc„ , where 
C„ is the cycle graph with n vertices, which loses positive definiteness when hard-thresholded 
at any level 77 > 0. 

Let Mn be the matrix considered in Theorem |3.3| 

\ 



We know from the proof of Theorem |3.3| that we can choose a and b such that M„ > 0, and 
M„ loses positive definiteness when a is set to zero. We will now prove that in addition, we 
can always choose & > 1 and |a| < 1 such that A/„ > and loses positive definiteness when 
a is set to 0. This will allow us to obtain the thresholded matrix by thresholding at level eg 
for some \a\ < eq < 1. The general result will follow by rescaling the matrix. 



2 


1 








... 





a 


1 


2 


1 





... 











1 


2 


1 


... 
















■■. 2 


1 

















1 


2 


b 


a 











... 


b 


2 



Following Theorem 3.3 we know that if we want M„ to lose positive definiteness when 
a is set to zero, we must choose 

b' = ^+e (3.2) 

for some e > small enough. Therefore, we can choose 6 > 1. Also, in order for Af„ to be 
positive definite, we must choose a between the two roots of the polynomial Pb{a) := p{a, b) 
when &^ takes the value prescribed in (3.2) (see the proof of Theorem 3.3). But notice that 
the maximum of pb is obtained when a — a* where 



(-1) 



ri+l 



1 



Therefore, if e is small enough, we have \a*\ < 1 and Pb{a*) > 0. With those choices of 
a — a* and b, the matrix Af„ is positive definite and loses positive definiteness if hard- 
thresholded at level \a\. The general case, for a given level 77 > 0, is obtained by considering 
the matrix (?7/|a|)M„. □ 



Remark: Theorem 3.6 proves that if the initial matrix A > has zeros according 
to a tree, and as the thresholding parameter 77 is continuously increased from to 1, the 
(piecewise continuous) path of thresholded matrices remain within the cone P+. 

Following Theorem |3.1[ it is natural to seek a characterization of the maximal set of 
matrices which will retain positive definiteness after thresholding with respect to a graph 
G. Let M{G) denote this set: 



MiG)^{M e¥+ ■.MGe¥+}. 
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According to Theorem 3.1 Ai{G) = P„ , i.e., every positive definite matrix retains positive 



definiteness when thresholded, if and only if G can be written as a union of complete 
disconnected components. But as discussed in Section 2.2 A4{G) always contains the set 
of diagonally dominant matrices, irrespective of the graph G. The following result shows 
that the set of diagonally dominant matrices is never maximal in the sense that, given any 
graph G, we can always find non diagonally dominant positive definite matrices which retain 
positive definiteness when thresholded with respect to G. We first prove a simple lemma 
required in the subsequent result. 

Lemma 3.7. Let G = (V, E) he a connected graph. Then there exists a vertex v £ V such 
that the subgraph of G induced by V\{v} is connected. 

Proof. Let u ^ V he any vertex of G and let w £ be a vertex at the maximum distance 
possible from u, i.e., every path from u to a vertex w € V\{u,v} contains less than or the 
same number of edges as the shortest path connecting uto v. We claim that the subgraph 
G* induced by is connected. Suppose to the contrary that this is not the case. Let 

w e y be a vertex that is not connected to u in G* . This means that every path from u to 
w in G passes through v. As a consequence, the distance between u and w in G is strictly 
greater that the distance between u and v. This contradicts the maximality of the distance 
between u and v. The graph G* must therefore be connected. □ 

Proposition 3.8. Let G — (V, E) he any undirected, connected graph with at least 3 vertices. 
Then there exists a matrix A — (aij) with the following properties: 

1. A is positive definite; 

2. A has no zeros; 

3. For every i, 



i.e. A is not diagonally dominant within any row; 

4-. Aq is positive definite; 

5. Aq is not diagonally-dominant. 

Proof. We will prove the result by induction on \V\. Assume first that \V\ = 3. It is easily 
verified that for any graph G, the following matrix satisfies the theorem: 




Now assume by the induction hypothesis that the theorem is true for \V\ = n — 1. Let G 
be a connected graph with n vertices. By Lemma |3.7[ there is a vertex v V such that the 
subgraph G* of G induced by V'\{w} is connected. By the induction hypothesis applied to 
G*, there exists a matrix j4„_i € M„_i(IR) satisfying properties (1) to (5) with respect to 
G*. Now consider the matrix 

/ ^1 \ 

XAn-l X2 



B = 



X2 
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We shall demonstrate that for appropriate values of a;i, . . . , a;„ and A, B will satisfy all the 
conditions of the theorem. 

One can first easily choose xi,. . . ,Xn such that B satisfies (2), (3) and (5). Indeed, let 
xi,. . . ,Xn be chosen such that 

1. Xi > for every i = 1, . . . ,n; 



Condition (2) is verified since Xj > for every i. Now, by the induction hypothesis, the 
matrix B is not diagonally dominant within the first n — 1 rows. Also, since \xi\ > \xn\, 
the matrix is not diagonally within the last row and so condition (3) holds. Finally, Bq 
is not diagonally dominant since, by the induction hypothesis, is not diagonally 

dominant. Therefore conditions (2), (3) and (5) hold for any such choice of a;i, . . . , a;„ and 
for any A > 0. 

We now proceed to show that we can adjust A in order to get (1) and (4) too. Notice 
that, for B and Bq to be positive definite, we only need to ensure that detB > and 
det Ba > 0. Let x = {xi,. . . Xn-iY be the vector containing the first n — 1 elements of the 
last column of B. Then 



which can be made to be positive for A > Ai > 0, say. The same is true for Bq, but with 

An^i and x thresholded. Therefore, for some A2 > 0, dot Bq > for A > A2. The result 



4 Thresholding by a chordal/decomposable graph 

In this section, we will characterize the set M{G) for certain classes of graphs. Recall 
that the proofs in the last sections hinges on constructing cycles of length n > 3. Hence a 
natural step in characterizing matrices which retain positive definitencss when thresholded 
with respect to a graph G is to consider graphs without induced cycles of length n > 3 or, 
more generally, graphs without induced cycles of length n > 4. Those graphs correspond 
to trees and chordal/decomposable graphs respectively. Characterizing A4{G) when G is 
chordal/decomposable is the topic of study in this section. We will see that, even in simple 
complete characterization may involve complex algebraic relations thus giving little 
hope of obtaining a general simple characterization of A4{G). 

Proposition 4.1. Let {A,B,C) be a decomposition of a graph G = {V,E), i.e., 

1. V = AV^BV^C; 

2. C separates A from B; 

3. C is complete. 

Let N be a positive definite matrix . Then M = Nq is positive definite iff 



2. X\ ^ Xn'-j 




follows by taking A = max(Ai, A2) in the construction of B. 



□ 
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2. M 



sue 



Mcc McB 
Mbc Mbb 



> 0; 



3. S1+S2-MCC > 0, where Si = Mcc-McaM^\Mac and S2 = Mcc-McbM^^^Mbc 
are Schur complements of Maa in MauC and Mbb in Mbuc respectively. 

Moreover, conditions (1) and (2) can be replaced by (1') and (2') where 

v. Maa > 

2'. Mbb > 0. 

Proof. Let us factor M according to the decomposition: 

Maa Mac 
M = I McA Mcc Mcb 
Mbc Mbb 



We know that M is positive definite iff 



Mbb > 



and 

Mauc - 
Let us develop the last term. 





Mcb 



^ M^l ( Mbc ) > 0. 



Ma 



uc 





Mcb 



Mgl ( Mbc 



Maa Mac 
Mca M, 

Ma, 



cc 





McbM^IMbc 



Ma 



Mca Mcc- Mcb M b^Mbc 



But the last matrix is positive definite iff Maa > and the Schur complement of Maa in 
this matrix given by 



cc 



McbM-'^Mbc 



McaM^aMac 



S1+S2- Mcc > 



where = Mcc — McaM^\Mac and 52 = Mcc — McbM^^Mbg are Schur complements 
of Maa in Mauc and Mbb in Mb\jc respectively. In summary, M is positive definite iff 
Maa > 0, Mbb > and + 52 — Mcc > 0. To conclude the proof, wc only need to prove 
that Maa > and Mbb > can be replaced by Mauc > and Mbuc > respectively. 

Note that if M > 0, then Mauc > and Mbuc > since they are principal submatrices 
of M. Therefore, M > implies the three conditions of the theorem. Conversely, if the 
three conditions of the theorem are satisfied, then in particular, Maa > and Mbb > 
as they are principal submatrices of Mauc and Mbuc- From the argument above, this 
together with the condition of the theorem 5i + 52 — Mcc > implies that M > 0. □ 



Remark: We note that another way to prove Proposition 4.1 is to factor M using a 
Cholesky type decomposition: 





( Maa 





M'cA 


M = 





Mbb 


M'cB 




V Mca 


Mcb 


Mcc 



Maa 







/ M-a\ 







/ Maa 











Mbb 


•) 


: 


Mb'b 


i) 




Mbb 





Mca 


Mcb 











V Mca 


Mcb 


I 
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When the graph G is chordal/decomposable, we can apply Proposition 4.1 recursively to 
characterize the class of the positive definite matrices that retain positive definiteness when 
thresholded. 



Theorem 4.2. Let G he a connected chordal/decomposable graph and let {Ci, . . . ,Gk) be 
a perfect order of its cliques. Let Hq — Ci U ■ ■ ■ U Gq and Sq ~ Gq D Hq^i (q = 2, . . . ,k). 
Moreover let Aq = Hq^i\Sq and Bq ~ Gq\Sq. Assume N is a positive definite matrix. 
Then M = Nq is positive definite if and only if for every 2 < q < k, 

^i'^+4'^-M5,S, >0 

where s['^ = Mg^s, - Ms^a^MX^a^Ma^s, and S^'^ = Mg^s, - Ms^b.M^Ib.Mb.s, are the 
Schur complements of Ma^a^ in Ma^us^.a^us, and of Mb^b^ in Ms^ub^,s,ub, respectively. 

Proof. We will prove the theorem by applying Proposition |4.1| recursively to different de- 
compositions of G that correspond to the perfect ordering (Ci, . . . , Ck). More specifically, 
we shall work backwards starting from Ck then to Cfc_i, etc., eventually leading up to G\. 

Consider first the decomposition {B^, A^, Sk) of G (see for example [5] Lemma 2.11). 
By an application of Proposition |4.1[ M > if and only if 

1. A/c.c. > 0; 

2. Mh,_,h,^, > 0; 

3. ) + 4^-) _ Ms,s, > 0. 

Notice that condition (1) is trivially verified since iV > and Gk is complete. Consider 
now the subgraph Gk-i induced by Hk-i- A decomposition of this subgraph is given by 
{Bk-iT Ak-i, Sk-i). By the same argument as above, the condition MH^_i.Hk-i > is 
equivalent to 

1. Mc._ic._i > 0; 

2. Mh,_,h,^, > 0; 

As a consequence, M > if and only if Mh^_^h^_^ > and 

^1'^ + 4'^ - M5,s, > (g = fc-l,fc). 

By applying the same reasoning to the graphs Gq induced by Hq (2 < q < k — 2) and 
working backwards, we obtain that Af > if and only if MhiHi > and 

+ ^(9) _ Af^^^^ > {2<q<k). 

Since Hi — Gi, the condition Mh^Hi > is trivially satisfied and so M > if and only if 

S['^ +8^'^ ^Ms,s,>0 {2<q<k). 

□ 
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Figure 1: Edge enumeration of a tree 



The next corohary shows that the conditions of the preceding theorem reduce to scalar 
conditions when the graph is a tree. 

Corollary 4.3. Let T be a tree with root vq. Let Ci — (pi, qi), i = 1, . . . ,k, be the edges of 
T labeled according to Figure^ i.e., starting from left and proceeding to the right at every 
depth of the tree. Note that pi is the parent of qi. Let N > and define M — Nq. For each 
j = 2, . . . , fc, define a scalar aj as folloius: 

where Aj = uiZi{Pii<li}\{Pj}- Also, let 

Then M > if and only if 
for every j = 2, . . . ,k. 

Proof. This follows from Theorem |4.2| by noting that the enumeration of the edges given 
above is in fact a perfect ordering of the cliques of T. □ 

Theorem |4.2| and Corollary |4.3| demonstrate that for decomposable graphs, or even sim- 
pler graphs like trees, characterizing the class of matrices retaining positive definiteness can 
lead to complex algebraic conditions. For a narrow class of graphs however, we can give 
a more explicit characterization of the matrices retaining positive definiteness. This is the 
case when the graph is "auto-similar" like a path for example. We first prove a lemma that 
is a key ingredient for our next result. 
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Lemma 4.4. Let An be a general matrix over a path of length n>3, given as follows 



a I 


(I I 






\ 


ai 


a-i 










ai 


0:3 










as 


0L4 a4 










dn-l Ctn-l ttn-l 










an—1 


J 



An — 



\ 

where the blank entries are zeros. Then the Schur complement of the lower right block in 
An, denoted cr„(l), is given by the following continued fraction: 



(7„(1) := ai 



a? 



a2 — 



Q!3 - 



a4 



(Xn-l 



Proof. We shall prove the result by induction. Assume first that n = 3. Then 

^3 = 



ai ai 
Oi a2 02 
02 as 



and the Schur complement of the lower right block in A3 is given by 
173(1) = ai - ( ai ) 
= ai - ( oi ) 



f a2 


a2 


)"'( 




V ^2 


"3 , 







1 



a2Ct3 — flj 



as -02 

— a2 Q.2 



ai 




= Ql 



a2a3 — a2 



(XT 



= ai - 



,2" 



a2 — 



as 



Now assume the expression is true for the (n — 1) x (n — 1) case and consider the n x n 
matrix as given in the statement of the lemma. The Schur complement of the lower right 
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block in An is given by 



a„(l) 



ai — ( ai 



) 



/ Q!2 02 
03 



as 

"4 



04 



On— 1 *^n— 1 a^ 



/ai\ 




V ; 



— ( ^1 











V ; 



where we have labeled A the lower right block of An- Now, notice that the righthand term 
of the last expression above is equal to of times the (1, 1) element of A~^. Recall that the 
upper left block of the inverse of a block matrix M denoted by 



M 



A B 
C D 



is given by the inverse of the Schur complement of D in M . Therefore, if S is the Schur 
complement of A in An, then 

(T„(l) = ai - a\S^^. 

But by the induction hypothesis, 



b = a2 5 



as 



Q4 



Oin-l 



On 

ar, 



and so the result follows. 



□ 



We now give algebraic conditions that characterize the class of p.d. matrices which retain 
positive definiteness when thresholded with respect to a path. We first note that the result 
below can be derived from first principles by using Lemma |4.4| and properties of positive 
definite matrices. Since a path is also a decomposable graph, we show that Proposition 
|4.1| can be useful in discovering such characterizations. We illustrate this idea in the next 
corollary. 

Corollary 4.5. Let G he a path of length n > 3 and let N be a n x n positive definite 
matrix. Let M = Nq given as in Lemma Then the thresholded matrix M is positive 

definite iff: 

V/c = l,...,n-2, (4.1) 



or equivalently, iff 



CT„(fc + 1) > ala^^ 

cr„(fc)>0 Vfc = l,. 



(4.2) 
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whe 



<7n{k) := ttfe - 



ak+2 



an 



Proof. We prove the result by induction on n. Suppose first that n = 3. Note that in this 
case, we only need to check (4.1 1 for k = 1. Since TV > 0, the matrix M = Nq is positive 



definite if and only if det M > 0. Now 



det M = ai{a2a3 — a\) 



(4.3) 



On the other hand, equation (4.1) can be expressed as 



^3(2) 



a, a 



1"! 



a2 



2 - 



det M. 



(4.4) 



Since > 0, the diagonal elements of M are positive and therefore the two conditions 



(4.3) and (4.4) are equivalent. This proves the result for n = 3. Now assume the result is 
true for every path with n — 1 vertices and let G be a path with n vertices. Let A = {1}, 
C = {2} and B = {3, . . . ,7i}. Then [A^C^B) is a decomposition of the path G and by 
Proposition [41] M > iff Mauc > 0, Afsuc > and 5i + ^2 - Mcc > where and S2 
are the Schur complements of M aa a nd Mbb in MauC and MbuC respectively. We have 
Si = a2 — afa^^ and by Lemma 4.4 S2 = cr„(2). Notice that Mauc > since M > and 
A U C is complete as it is not affected by the thresholding. Therefore, Af > iff Mbuc > 
and Si+ S2 — Mcc = a2 — afa^^ + o-„(2) — a2 = an{2) — ala^^ > 0. The latter condition 



is equivalent to equation (4.1 1 for k — 1. Now, Mbuc is the same matrix as M but for a 
path of n — 1 points. Therefore, by the induction hypothesis, the matrix Mbuc is positive 
definite iff 



o-„(/s 



1) - Ofe^fc^ 



> 



V/c = 2, 



,n- 2. 



As a consequence, the matrix M — Nc is positive definite if and only if 



cr„(fc + 1) > a^ttj. - 



yk = 1, 



and the result follows. The preceding condition can be simplified to cr„(fc) > for k = 
1, . . . ,n ~ 2 since 

„2 

(T„(fc) = ak 



a„{k + l)' 

This proves the equivalence between conditions (|4.1|) and (|4.2| and concludes the proof. □ 
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